Fusion of Perceptual Cues using Covariance Estimation

The paradigm of perceptual fusion provides robust solutions to computer vision problems. By combining the outputs of multiple vision modules, the assumptions and constraints of each module are factored out to result in a more robust system overall. The integration of different modules can be regarded as a form of data fusion. To this end, we propose a framework for fusing different information sources through estimation of covariance from observations. The framework is demonstrated in a face and 3D pose tracking system that fuses similarity-to-prototypes measures and skin colour to track head pose and face position. The use of data fusion through covariance introduces constraints that allow the tracker to robustly estimate head pose and track face position simultaneously.